Similar case retrieval apparatus, similar case retrieval method, non-transitory computer-readable storage medium, similar case retrieval system, and case database

ABSTRACT

A similar case retrieval apparatus includes: a lesion portion acquirer that acquires partial images including lesion portion images, an image feature extractor that extracts image features of each of the plurality of partial images; a location information acquirer that acquires location information of each of the partial images; a lateral position determiner that determines the right organ or the left organ in which each of the lesion portions exists based on the location information; a unilateral distribution identifier that determines whether or not a distribution of the lesion portions is a unilateral distribution; and a similar case retriever that retrieves case data from a case database including both case data for the unilateral distribution in the right organ and case data for the unilateral distribution in the left organ when the unilateral distribution identifier identifies that the distribution of the lesion portions is the unilateral distribution.

BACKGROUND

1. Technical Field

The present disclosure relates to similar case retrieving techniques.

2. Description of the Related Art

Such a conventional apparatus is known that provides, as a similar case,the same mammographic image every time when the same region of interestis specified, even if the manner of specifying the region of interestvaries depending on a doctor, who is a user (see PTL 1 of PatentLiterature).

CITATION LIST Patent Literature

-   PTL 1: Unexamined Japanese Patent Publication No. 2010-133

Non-Patent Literatures

-   NPL 1: “Improvement of Tumor Detection Performance in Mammograms by    Feature Selection from a Large Number of Features and Proposal of    Fast Feature Selection Method” by M. Nemoto, A. Shimizu, Y.    Hagihara, H. Kobata, S. Nawano, The IEICE (Institute of Electronics,    Information and Communication Engineers of Japan) Transactions    (Japanese Edition) D-II, Vol. J88-D-II, No. 2, pp. 416-426, February    2005-   NPL 2: “Extraction of lung region from 3D thoracic CT images with    diffuse pulmonary diseases by use of graph cut and statistical    atlas” by R. Urayama, R. Xu, Y. Hirano, S. Kido, The Technical    Report of The Proceeding of The Institute of Electronics,    Information and communication Engineers of Japan, Medical Imaging    (MI), Vol. 112, No. 411, pp. 135-138, January 2013

However, similar case retrieval cannot appropriately be performed withthe configuration disclosed by PTL 1 when plural lesion portions existin a pair of right and left organs such as a pair of lungs.

SUMMARY

One non-limiting and exemplary embodiment provides a similar caseretrieval apparatus that can appropriately perform a similar caseretrieval when plural lesion portions exist in a pair of organs.

In one general aspect, the techniques disclosed here feature a similarcase retrieval apparatus that includes: a lesion portion acquirer thatacquires a plurality of partial images from a plurality ofinterpretation target images related to a pair of right and left organs,the plurality of partial images including a plurality of lesion portionimages of lesion portions, each of the plurality of partial imagesincluding a lesion portion image that is one of the plurality of lesionportion images; an image feature extractor that extracts one or moreimage features of each of the plurality of partial images; a locationinformation acquirer that acquires location information of each of theplurality of partial images; a lateral position determiner thatdetermines the right organ or the left organ in which each of the lesionportions exists based on the location information; a unilateraldistribution identifier that determines, based on the determinationresults by the lateral position determiner, whether or not adistribution of the lesion portions is a unilateral distribution inwhich the lesion portions are distributed unilaterally in either theright organ or the left organ; and a similar case retriever thatretrieves case data from a case database including both case data forthe unilateral distribution in the right organ and case data for theunilateral distribution in the left organ, based on the one or moreimage features extracted by the image feature extractor and imagefeatures of images contained in the case database, when the unilateraldistribution identifier identifies that the distribution of the lesionportions is the unilateral distribution, an image feature of an imagecontained in each of the retrieved case data being similar to at leastone of the one or more image features extracted by the image featureextractor.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

It should be noted that general or specific embodiments may beimplemented as a system, an apparatus, a method, an integrated circuit,a computer program, or a computer-readable storage medium, or may berealized as any combination of a system, an apparatus, a method, anintegrated circuit, a computer program and a storage medium. Thecomputer-readable storage medium may include a non-volatile storagemedium such as a CD-ROM (Compact Disc-Read Only Memory).

The similar case retrieval apparatus in accordance with the presentdisclosure can appropriately perform similar case retrieval when plurallesion portions exist in a pair of organs.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a functional configuration of asimilar case retrieval apparatus in accordance with an exemplaryembodiment;

FIG. 2 is a diagram showing an example of case data registered in a casedatabase;

FIG. 3 shows a schematic diagram of lesions in a lung area, a diagramshowing an example of tomography image, and a diagram showing anotherexample of tomography image;

FIG. 4 is a diagram showing an example of acquiring location informationof a partial image containing a lesion portion;

FIG. 5 is a flowchart showing an overall processing flow that isperformed by the similar case retrieval apparatus in accordance with theexemplary embodiment;

FIG. 6 is a flowchart showing a detailed processing flow of a lateralposition determination;

FIG. 7 is a diagram showing an example of recording a lateral positiondetermination result;

FIG. 8 is a flowchart showing a detailed processing flow of a similarcase retrieval;

FIG. 9 is a diagram showing an example of combining partial images eachcontaining a lesion portion of an interpretation target image andpartial images each containing a lesion portion of case data;

FIG. 10A is a diagram showing an example of query case having aunilateral distribution;

FIG. 10B is a diagram showing an example of retrieved similar case;

FIG. 10C is a diagram showing another example of retrieved similar case;

FIG. 11A is a schematic diagram showing lesions in a lung area;

FIG. 11B is a diagram showing an example of tomography images ofmultiple lesions;

FIG. 11C is a diagram showing an example of tomography images of asolitary lesion; and

FIG. 12 is a flowchart for showing a position of the similar caseretrieval.

DETAILED DESCRIPTION Underlying Knowledge Forming Basis of the PresentDisclosure

Recently, in the field of diagnostic imaging, digitalization ofphotographed images and image interpretation reports has beenprogressed, and it is easy for doctors to share a large amount of dataitems. As one of secondary uses of these data items, such an effort isexpected that supports a decision-making concerning diagnosis bypresenting a similar case with respect to an interpreting image, whichis an object to be diagnosed, from stored data items.

In retrieving similar cases, it is necessary to retrieve a case that issimilar to a lesion in a specified region of interest in the locationand distribution of the lesion as well as the morphology of the lesion.Because, diagnosis and treatment policy with respect to lesions maychange depending on the locations and/or the distributions of thelesions even if they show the same morphology. As a conventional similarcase retrieving technique considering both the lesion morphology and thelesion location, PTL 1 discloses a technique that retrieves similarcases from past cases by using location information and breast densityinformation of a lesion candidate detected from a breast image.According to the method disclosed by PTL 1, it is possible to retrievecases that are similar in the lesion location as well as the lesionmorphology in the region of interest.

In a case of an image diagnosis of a lung field, for example, thedistribution of multiple lesions contributing to diagnosis can bebroadly categorized into the “unilateral” distribution and the“bilateral” distribution. The unilateral distribution is a state inwhich plural lesion portions exist in either one of the right lung andthe left lung, and the bilateral distribution is a state in which plurallesion portions exist in both of the right lung and the left lung. Inother words, when a lesion having a unilateral distribution is input asa search query for similar case retrieval, it is necessary to retrievecases that are similar in image morphology to the query image from allcases having the unilateral distribution without discriminating theright lung and the left lung.

In the method disclosed by PTL 1, however, cases in the same locationare retrieved with a high priority. Accordingly, when a case having aunilateral distribution in one of a pair of lungs is input as a searchquery, the group of cases each having a unilateral distribution in theother of the pair of lungs (the left lung if the input case is in theright lung) are excluded from the search objects. As a result, even if asimilar case having similar image morphology exists in the group ofcases each having a unilateral distribution in the other of the pair oflungs, this similar case cannot be retrieved. In other words, the methoddisclosed by PTL 1 cannot appropriately retrieve similar cases.

This problem is not limited to the cases of lungs, and exists in thecases of the other pairs of organs such as brains, breasts, and kidneys.

Therefore, a first aspect of the present disclosure provides a similarcase retrieval apparatus that includes: a lesion portion acquirer thatacquires a plurality of partial images from a plurality ofinterpretation target images related to a pair of right and left organs,the plurality of partial images including a plurality of lesion portionimages of lesion portions, each of the plurality of partial imagesincluding a lesion portion image that is one of the plurality of lesionportion images; an image feature extractor that extracts one or moreimage features of each of the plurality of partial images; a locationinformation acquirer that acquires location information of each of theplurality of partial images; a lateral position determiner thatdetermines the right organ or the left organ in which each of the lesionportions exists based on the location information; a unilateraldistribution identifier that determines, based on the determinationresults by the lateral position determiner, whether or not adistribution of the lesion portions is a unilateral distribution inwhich the lesion portions are distributed unilaterally in either theright organ or the left organ; and a similar case retriever thatretrieves case data from a case database including both case data forthe unilateral distribution in the right organ and case data for theunilateral distribution in the left organ, based on the one or moreimage features extracted by the image feature extractor and imagefeatures of images contained in the case database, when the unilateraldistribution identifier identifies that the distribution of the lesionportions is the unilateral distribution, an image feature of an imagecontained in each of the retrieved case data being similar to at leastone of the one or more image features extracted by the image featureextractor.

This makes it possible to perform a similar case retrieval consideringwhether or not a distribution of plural lesion portions existing in apair of organs such as lungs is the unilateral distribution. That is, ina case of the unilateral distribution, a similar case retrieval can beperformed so that cases to be retrieved include both of cases which areunilaterally distributed in a right organ and cases which areunilaterally distributed in a left organ. Consequently, it is possibleto appropriately perform similar case retrieval.

A second aspect of the present disclosure provides the similar caseretrieval apparatus according to the first aspect, wherein the pair oforgans are a pair of lungs.

A third aspect of the present disclosure provides the similar caseretrieval apparatus according to the first aspect, wherein, when thelesion portion acquirer determines that a first lesion portion containedin an acquired first partial image and a second lesion portion containedin an acquired second partial image are included in a solitary lesion,the lesion portion acquirer does not notify the location informationacquirer of location information of the first partial image and locationinformation of the second partial image, and wherein the plurality ofpartial images includes the acquired first partial image and theacquired second partial image.

This allows only multiple lesions to be the objects that are checked toidentify the unilateral distribution. Accordingly, it is possible toprevent reduction of search accuracy.

A fourth aspect of the present disclosure provides the similar caseretrieval apparatus according to the third aspect, wherein the lesionportion acquirer determines a lesion as the solitary lesion when apercentage of an area having a normal CT value is equal to or lower thana predetermined threshold value in a tomography image of a tomographicslice plane of one of the pair of organs between a first tomographicslice plane of the organ identified by a tomography image containing thefirst partial image and a second tomographic slice plane of the of thepair of organs identified by a tomography image containing the secondpartial image.

A fifth aspect of the present disclosure provides the similar caseretrieval apparatus according to the first aspect, further including anoutput that outputs the case data retrieved by the similar caseretriever to an outside.

A sixth aspect of the present disclosure provides the similar caseretrieval apparatus according to the first aspect, further including adatabase updater that registers, in the case database, data includingthe interpretation target image and information that is theidentification result by the unilateral distribution identifier and thatindicates whether or not a distribution of a lesion is the unilateraldistribution.

This makes it possible to sequentially store the retrieved cases in thedatabase, and thus to automatically increase the number of cases to besearched.

A seventh aspect of the present disclosure provides a similar caseretrieval method that includes: acquiring a plurality of partial imagesfrom a plurality of interpretation target images related to a pair ofright and left organs, the plurality of partial images including aplurality of lesion portion images of lesion portions, each of theplurality of partial images including a lesion portion image that is oneof the plurality of lesion portion images; extracting one or more imagefeatures of each of the plurality of partial images; acquiring locationinformation of each of the plurality of partial images; determining theright organ or the left organ in which each of the lesion portionsexists based on the location information; determining, based on thedetermination results in the determining the right organ or the leftorgan, whether or not a distribution of the lesion portions is aunilateral distribution in which the lesion portions are distributedunilaterally in either the right organ or the left organ; and retrievingcase data from a case database including both case data for theunilateral distribution in the right organ and case data for theunilateral distribution in the left organ, based on the one or moreimage features extracted in the extracting one or more image featuresand image features of images contained in the case database, when theunilateral distribution identifier identifies that the distribution ofthe lesion portions is the unilateral distribution, an image feature ofan image contained in each of the retrieved case data being similar toat least one of the one or more image features extracted in theextracting one or more image features.

A eighth aspect of the present disclosure provides a non-transitorycomputer-readable storage medium storing a program that causes acomputer to execute a similar case retrieval method, the similar caseretrieval method that includes: acquiring a plurality of partial imagesfrom a plurality of interpretation target images related to a pair ofright and left organs, the plurality of partial images including aplurality of lesion portion images of lesion portions, each of theplurality of partial images including a lesion portion image that is oneof the plurality of lesion portion images; extracting one or more imagefeatures of each of the plurality of partial images; acquiring locationinformation of each of the plurality of partial images; determining theright organ or the left organ in which each of the lesion portionsexists based on the location information; determining, based on thedetermination results in the determining the right organ or the leftorgan, whether or not a distribution of the lesion portions is aunilateral distribution in which the lesion portions are distributedunilaterally in either the right organ or the left organ; and retrievingcase data from a case database including both case data for theunilateral distribution in the right organ and case data for theunilateral distribution in the left organ, based on the one or moreimage features extracted in the extracting one or more image featuresand image features of images contained in the case database, when theunilateral distribution identifier identifies that the distribution ofthe lesion portions is the unilateral distribution, an image feature ofan image contained in each of the retrieved case data being similar toat least one of the one or more image features extracted in theextracting one or more image features.

A ninth aspect of the present disclosure provides a similar caseretrieval system that includes: a similar case retrieval apparatus; anda case database including a plurality of images, wherein the similarcase retrieval apparatus comprises: a lesion portion acquirer thatacquires a plurality of partial images from a plurality ofinterpretation target images related to a pair of right and left organs,the plurality of partial images including a plurality of lesion portionimages of lesion portions, each of the plurality of partial imagesincluding a lesion portion image that is one of the plurality of lesionportion images; an image feature extractor that extracts one or moreimage features of each of the plurality of partial images; a locationinformation acquirer that acquires location information of each of theplurality of partial images; a lateral position determiner thatdetermines the right organ or the left organ in which each of the lesionportions exists based on the location information; a unilateraldistribution identifier that determines, based on the determinationresults by the lateral position determiner, whether or not adistribution of the lesion portions is a unilateral distribution inwhich the lesion portions are distributed unilaterally in either theright organ or the left organ; and a similar case retriever thatretrieves case data from the case database including both case data forthe unilateral distribution in the right organ and case data for theunilateral distribution in the left organ, based on the one or moreimage features extracted by the image feature extractor and imagefeatures of the plurality of images contained in the case database, whenthe unilateral distribution identifier identifies that the distributionof the lesion portions is the unilateral distribution, an image featureof an image contained in each of the retrieved case data being similarto at least one of the one or more image features extracted by the imagefeature extractor.

A tenth aspect of the present disclosure provides a case database thatincludes a plurality of images, wherein a similar case retrievalapparatus uses the case data base, and wherein the similar caseretrieval apparatus comprises: a lesion portion acquirer that acquires aplurality of partial images from a plurality of interpretation targetimages related to a pair of right and left organs, the plurality ofpartial images including a plurality of lesion portion images of lesionportions, each of the plurality of partial images including a lesionportion image that is one of the plurality of lesion portion images; animage feature extractor that extracts one or more image features of eachof the plurality of partial images; a location information acquirer thatacquires location information of each of the plurality of partialimages; a lateral position determiner that determines the right organ orthe left organ in which each of the lesion portions exists based on thelocation information; a unilateral distribution identifier thatdetermines, based on the determination results by the lateral positiondeterminer, whether or not a distribution of the lesion portions is aunilateral distribution in which the lesion portions are distributedunilaterally in either the right organ or the left organ; and a similarcase retriever that retrieves case data from the case database includingboth case data for the unilateral distribution in the right organ andcase data for the unilateral distribution in the left organ, based onthe one or more image features extracted by the image feature extractorand image features of the plurality of images contained in the casedatabase, when the unilateral distribution identifier identifies thatthe distribution of the lesion portions is the unilateral distribution,an image feature of an image contained in each of the retrieved casedata being similar to at least one of the one or more image featuresextracted by the image feature extractor.

Explanation of Terms

Terms used in the following exemplary embodiment will be explained.

The “image features” include features regarding a shape of an organ or alesion portion in a medical image, and features regarding brightnessdistribution in a medical image. For example, NPL 1 of the Non-PatentLiterature discloses 490 kinds of features (feature information) as theimage features. In the present disclosure also, the image features to beused include several tens to several hundred kinds of image featureswhich are predetermined for each medical image photographing apparatus(modality) used to photograph a medical image and for each target organ.

Further, the medical images in the present disclosure include ultrasoundimages, CT (Computed Tomography) images or MRI (Magnetic ResonanceImaging) images.

The “unilateral distribution” is a state in which a plurality of lesionsexist in either one of a pair of organs, and the “bilateraldistribution” is a state in which a plurality of lesions exist in bothof a pair of organs. The “multiple lesions” are a plurality of lesionsexisting at different locations in an organ area, and the “solitarylesion” is a single lesion existing at an arbitrary location in an organarea.

Exemplary Embodiment

Hereinafter, description will be made by taking a pair of lungs as anexample of the pair of organs.

Configuration of Apparatus

FIG. 1 is a block diagram showing a functional configuration of similarcase retrieval apparatus 100 in accordance with an exemplary embodiment.

Similar case retrieval apparatus 100 in FIG. 1 is an apparatus whichretrieves a similar case data according to an image interpretationresult by an image interpreter from case database 101 in which case datacontaining medical images have been registered. As shown in FIG. 1,similar case retrieval apparatus 100 includes lesion portion acquiringunit 102, image feature extracting unit 103, location informationacquiring unit 104, lateral position determining unit 105, unilateraldistribution identifying unit 106, similar case retrieving unit 107, andoutput unit 108. Database updating unit 109 will be described later.Database updating unit 109 may be omitted.

Hereinafter, details of each component of case database 101 and similarcase retrieval apparatus 100 which are illustrated in FIG. 1 will bedescribed in order.

Case database 101 is a storage device including, for example, a harddisk and a memory, and stores therein case data including interpretationimage data providing an image interpreter with medical images, and imageinterpretation information corresponding to the interpretation imagedata. Here, the interpretation image data are image data used for imagediagnosis and stored in an electronic medium. Further, the imageinterpretation information is information associated with theinterpretation image data, and includes documentation data such aspatient information and retrieval results.

FIG. 2 shows an example of case data registered in case database 101.Lesion region 21 is set in interpretation image data 20 of an organ.Interpretation information 22 includes patient ID 23, image ID 24, imagegroup ID 25 (indicating a group of plural images obtained by one scan ina case of CT images), and additional information including, for example,image features 26 of lesion region 21, and lesion distributioninformation 27 indicating whether the state of the lesion is theunilateral distribution or the bilateral distribution.

Lesion portion acquiring unit 102 acquires partial images eachcontaining a lesion portion from interpretation target images, or CTimages of a lung area here. Each partial image containing a lesionportion is an image of a specific region in an interpretation targetimage. Lesion portion acquiring unit 102 outputs these acquired partialimages each containing a lesion portion to image feature extracting unit103 and location information acquiring unit 104. Here, a partial imagecontaining a lesion portion may be an entire interpretation targetimage.

An example of acquiring partial images each containing a lesion portionis shown in FIG. 3. When multiple lesions A and B exist in lung area LAas shown in (a) in FIG. 3, tomography images I1 and I2 as respectivelyshown in (b) and (c) in FIG. 3 are taken in CT image diagnosis. Withrespect to each of tomography images I1 and I2, a user specifies imageregion 33 containing a lesion portion. Lesion portion acquiring unit 102acquires information of image region 33 containing the specified lesionportion, or of a partial image region containing the lesion portion,such as coordinate data of a rectangular frame.

Image feature extracting unit 103 extracts one or more image featuresfrom each of the partial images each containing a lesion portionacquired by lesion portion acquiring unit 102, and outputs the extractedimage features to similar case retrieving unit 107.

Location information acquiring unit 104 acquires location information ofthe partial images each containing a lesion portion acquired by lesionportion acquiring unit 102, and outputs the acquired locationinformation to lateral position determining unit 105. Specifically, thelocation information may be the coordinate data of the partial imageseach containing a lesion portion. For example, as shown in FIG. 4,center coordinates 40 of each partial image containing a lesion portionmay be acquired as the location information.

Lateral position determining unit 105 determines in which of the rightlung area and the left lung area each of the lesion portions existsbased on the location information acquired from location informationacquiring unit 104. The determination results are output to unilateraldistribution identifying unit 106. Specific determination method will bedescribed later.

Unilateral distribution identifying unit 106 identifies, from thedetermination results by lateral position determining unit 105, whetherthe state of the lesions is a unilateral distribution in which theplural lesions exist in only one of the right lung and the left lung.This identification result is output to similar case retrieving unit107. Specific identification method will be described later.

Similar case retrieving unit 107 retrieves, from case database 101, casedata each showing a state similar to that of the interpretation targetimage, by comparing the image features extracted by image featureextracting unit 103 to image features extracted from medical imagescontained in case data registered in case database 101. In thisretrieval operation, when unilateral distribution identifying unit 106identifies the state of the lesion contained in the interpretationtarget image as the unilateral distribution, similar case retrievingunit 107 retrieves, among the case data registered in case database 101,both of case data each showing the unilateral distribution in the rightlung and case data each showing the unilateral distribution in the leftlung. Specific retrieval method will be described later.

Output unit 108 outputs the case data obtained by similar caseretrieving unit 107 to the outside of similar case retrieval apparatus100, such as an output medium.

Next, an operation of similar case retrieval apparatus 100 configured asabove will be described.

Operation

FIG. 5 is a flowchart showing an overall processing flow that isperformed by similar case retrieval apparatus 100 shown in FIG. 1.

First, lesion portion acquiring unit 102 acquires a plurality of partialimages each containing a lesion portion from a plurality of lung CTimages, which are objects to be interpreted, and notifies image featureextracting unit 103 and location information acquiring unit 104 of theacquired partial images each containing a lesion portion (step S101).Here, one CT image contains one partial image containing a lesionportion.

However, one CT image may contain plural partial images each containinga lesion portion.

Image feature extracting unit 103 extracts one or more image featuresfrom each of the partial images each containing a lesion portionobtained from lesion portion acquiring unit 102, and notifies similarcase retrieving unit 107 of the extracted image features (step S102).

Also, location information acquiring unit 104 acquires locationinformation of the partial images each containing a lesion portionobtained from lesion portion acquiring unit 102, and notifies lateralposition determining unit 105 of the acquired location information (stepS103).

Next, lateral position determining unit 105 determines in which of theright lung and the left lung each of the plurality of lesions is locatedbased on the location information of the plurality of partial imageseach containing a lesion portion, the location information obtained fromlocation information acquiring unit 104, and notifies unilateraldistribution identifying unit 106 of the determined results (step S104).

FIG. 6 is a flowchart showing a detailed processing flow of step S104,or the lateral position determination processing.

First, lateral position determining unit 105 obtains the locationinformation of the partial images each containing a lesion portion fromlocation information acquiring unit 104. The obtained locationinformation of each partial image containing a lesion portion may, forexample, be coordinate data such as center coordinates orcenter-of-gravity coordinates of the partial image (step S201).

Next, lateral position determining unit 105 selects one partial imagecontaining a lesion portion from the plurality of partial images eachcontaining a lesion portion which have been obtained in step S201 (stepS202).

Next, lateral position determining unit 105 extracts a lung area from atomography image containing the partial image which has been selected instep S202 (step S203). As a method of extracting the lung area, forexample, the image processing method as disclosed by NPL 2 of theNon-Patent Literatures may be used to automatically extract the lungarea.

Next, with respect to the partial image selected in step S202, lateralposition determining unit 105 determines in which of the right side andthe left side of the lung area extracted in step S203 the locationindicated by the coordinate data of the partial image obtained in stepS201 exists (step S204). Specifically, it may be determined, withrespect to the partial image selected in step S202, in which side of theextracted lung area the location indicated by the coordinates obtainedin step S201 exists.

Next, lateral position determining unit 105 records the determinationresult obtained in step S204 (step S205). As a recording method, forexample, sets of lesion portion ID 70 for identifying each lesionportion and right/left information 71 which is the determination resultmay be recorded in a list form as shown in FIG. 7.

Next, lateral position determining unit 105 checks whether or not allpartial images each containing a lesion portion have been selected.Then, the processing proceeds to step S207 if all partial images havebeen selected, and returns to step S202 if there remain some partialimages each containing a lesion portion which have not yet been selected(step S206).

In step S207, lateral position determining unit 105 notifies unilateraldistribution identifying unit 106 of the right/left determinationresults recorded in step S205.

By performing the processing as shown in FIG. 6, it is possible in stepS104 to determine in which of the right lung area and the left lung areaeach of the plurality of lesion portions exist.

Referring back to FIG. 5, unilateral distribution identifying unit 106identifies whether the lesion distribution is the unilateraldistribution, in which a plurality of lesion portions exist in eitherone of the right lung and the left lung, from the determination resultsobtained from lateral position determining unit 105, and notifiessimilar case retrieving unit 107 of the identified result (step S105).Here, the identification method may, for example, be such that thedistribution is identified as the unilateral distribution if all of theright/left determination results obtained from lateral positiondetermining unit 105 are the same. For example, the example shown inFIG. 7 is not the unilateral distribution, because three lesion portionsexist in the left lung and one lesion portion exists in the right lung.On the other hand, if all of the lesion portions exist in the left lung,the distribution is identified as the unilateral distribution. However,even in a case where not all lesion portions exist in one of a rightorgan and a left organ, the distribution may be identified as theunilateral distribution if the lesion portions exist in one of the pairof organs unilaterally to a certain extent.

Next, similar case retrieving unit 107 retrieves case data similar tothe state indicated by the interpretation target image from casedatabase 101 by comparing image features extracted from medical imagescontained in case data registered in case database 101 to image featuresextracted by image feature extracting unit 103 in step S102. At thistime, if the unilateral distribution has been identified by unilateraldistribution identifying unit 106 in step S105, case data of the bothunilateral distributions, or both the case data of the right lung andthe case data of the left lung, are searched (step S106).

FIG. 8 is a flowchart showing a detailed processing flow of step S106,or of similar case retrieval processing. Here, it is assumed that theunilateral distribution has been identified in step S105.

First, similar case retrieving unit 107 selects case data each showingthe unilateral distribution from case database 101 (step S301). Next,similar case retrieving unit 107 arbitrarily selects combinations of theplurality of partial images each containing a lesion portion obtained instep S101 of FIG. 5 and the plurality of partial images each containinga lesion portion contained in the case data selected in step S301 (stepS302).

FIG. 9 shows an example of combination of partial images each containinga lesion portion of an interpretation target image and partial imageseach containing a lesion portion of case data. In the example shown inFIG. 9, two partial images (q1, q2) each containing a lesion portion isobtained from the interpretation target image in step S101, and twopartial images (d1, d2) each containing a lesion portion is obtainedfrom the case data selected in step S301. In this state, there are twopossible combinations of the partial images each containing a lesionportion of the interpretation target images and the partial images eachcontaining a lesion portion of the images of the case data, i.e., acombination (q1×d1, q2×d2) and a combination (q1×d2, q2×d1). In stepS302, an arbitrary combination (q1×d2, q2×d1 in FIG. 9) is selected fromthese combinations.

Next, similar case retrieving unit 107 calculates image similaritybetween the partial images each containing a lesion portion in thecombination selected in step S302 (step S303). A specific method ofcalculating the similarity, for example, may calculate each cosinedistance between an image feature vector which is a vectorrepresentation of an image feature of a partial image containing alesion portion of the interpretation target image and an image featurevector of a partial image containing a lesion portion contained in thecase data, and calculates a sum of the calculated cosine distances as asimilarity. In the example shown in FIG. 9, for example, a sum of acosine distance between partial image q1 and partial image d2 eachcontaining a lesion portion and a cosine distance between partial imageq2 and partial image d1 each containing a lesion portion is calculatedas a similarity.

Next, similar case retrieving unit 107 determines whether or not allcombinations of the partial images each containing a lesion portion havebeen selected (step S304). The processing proceeds to step S305 when allcombinations have been selected, and returns to step S302 when thereremain some combinations which have not yet been selected. Then, in stepS305, similar case retrieving unit 107 identifies a maximum similarityfrom the similarities calculated in step S303, and records thecombination of the lesions providing the maximum similarity and thecalculated maximum similarity.

Then, similar case retrieving unit 107 determines whether or not allcase data recorded in case database 101 and related to the unilateraldistribution have been selected (step S306). The processing proceeds tostep S307 when all such case data have been selected, and returns tostep S301 when there remain some case data which have not yet beenselected.

Finally, similar case retrieving unit 107 selects, from the case datarecorded in step S305, case data each having a maximum similarity thatis equal to or larger than a predetermined threshold value, and outputsthe selected case data to output unit 108 (step S307).

By performing the processing as shown in FIG. 8, in step S106, it ispossible to retrieve case data each containing an image similar to theinterpretation target image from case database 101.

Referring back to FIG. 5, finally, output unit 108 outputs the case dataobtained from similar case retrieving unit 107 to an external outputmedium, for example (step S107).

Here, advantageous effects of discriminating the unilateral distributionin the case of performing the similar case retrieval of multiple lesionswill be described. As described above, in the image diagnosis withrespect to multiple lesions of a pair of organs such as a pair of lungs,the diagnosis result often varies depending on whether the lesiondistribution is the unilateral distribution or the bilateraldistribution. Accordingly, with respect to a lesion in a state of theunilateral distribution, it is preferable to retrieve similar cases frompast cases each being in a state of the unilateral distribution.

FIG. 10A, FIG. 10B, and FIG. 10C show an example of similar casesretrieved with respect to a query case having the unilateraldistribution. With respect to a query case shown in FIG. 10A, similarcases shown in FIG. 10B and FIG. 10C are retrieved. In the conventiontechniques, such lesion is retrieved that is similar to a lesion to bediagnosed in the lesion distribution location as well as the imagemorphology of the lesion. Accordingly, with respect to the query case ofthe right lung shown in FIG. 10A, similar cases in which lesions aredistributed in the right lung (the case shown in FIG. 10B) may bepreferentially retrieved. On the other hand, the case in which lesionsare distributed in the left lung (the case shown in FIG. 10C) may beless possible to be presented to the user, although it is an importantreference case having the unilateral distribution, because it may bedetermined as low in the similarity and thus lowered in the retrievalpriority, for the reason that the lesion distribution location is theleft, or different from the right.

On the other hand, according to the present exemplary embodiment,similar case retrieval in a case of the unilateral distribution isperformed without discriminating the unilateral distribution in theright lung and the unilateral distribution in the left lung.Consequently, with respect to the query case shown in FIG. 10A, casesthat are similar in image morphology will preferentially be retrieved,including the cases in which the lesions are distributed in the leftlung as shown in FIG. 10C as well as the cases in which the lesions aredistributed in the right lung as shown in FIG. 10B. Accordingly, it ispossible to retrieve optimum similar cases from all cases that areeffective to diagnosis.

According to the present exemplary embodiment, as described above,similar case retrieval apparatus 100 identifies whether or not aplurality of partial images each containing a lesion portion existing inlungs show a state of the unilateral distribution, and, in a case wherethe unilateral distribution is identified, searches case data eachshowing the unilateral distribution from both of the case data of theright lung and the case data of the left lung as objects for similarcase retrieval. This makes it possible to retrieve appropriate similarcases.

Incidentally, even when a plurality of partial images each containing alesion portion are acquired by lesion portion acquiring unit 102, thesepartial images each containing a lesion portion is not always showmultiple lesions.

FIG. 11A, FIG. 11B, and FIG. 11C show an example of determining aplurality of lesion portions. When two lesions A and B exist in lungarea LA as shown in FIG. 11A, partial images respectively containing thetwo lesion portions are acquired from tomography images I1 and I2 withrespect to two lesions A and B as shown in FIG. 11B. The acquired pluralpartial images each containing a lesion portion show multiple lesions.However, in a case where only lesion A exists, for example, such a casemay possibly occur that two partial images each containing the lesionportion are acquired from tomography images I1 and I3 as shown in FIG.11C. In this case, a plurality of partial images each containing alesion portion are acquired, although the existing lesion is a solitarylesion. The solitary lesion and the multiple lesions are findings thatshow different distributions from each other. Therefore, if these areretrieved as included in the same distribution, a case that is notintended by the user may be retrieved.

Therefore, when lesion portion acquiring unit 102 determines thatacquired plural partial images each containing a lesion portion indicatea solitary lesion, lesion portion acquiring unit 102 excludes thissolitary lesion from the objects to be checked for identification of theunilateral distribution. That is, lesion portion acquiring unit 102 doesnot send the information of the solitary lesion to location informationacquiring unit 104. Accordingly, only multiple lesions can be theobjects to be checked for the unilateral distribution identification, sothat it is possible to prevent reduction of retrieval accuracy.

As a method of determining the solitary lesion, it may be determinedthat a lesion is a solitary lesion if the percentage of an area showinga normal value in an area between plural lesion portions is equal to orlower than a predetermined threshold value. For example, in a case of CTimages, it may be determined that a lesion is a solitary lesion if thepercentage of an area showing a CT value within a normal range in animage region (e.g., tomography image J shown in FIG. 11A) between pluraltomography images each containing a lesion portion (e.g., tomographyimage I1 and tomography image I3 shown in FIG. 11A) is equal to or lowerthan a predetermined threshold value. Here, the CT value is a numericalexpression of x-ray absorption in human body, and is expressed by arelative value to the value 0 of water (unit: HU).

Also, similar case retrieval apparatus 100 in accordance with thepresent exemplary embodiment may further includes database updating unit109 that registers in case database 101 the data containing theinterpretation target image and the identification result by unilateraldistribution identifying unit 106 when partial images each containing alesion portion are acquired by lesion portion acquiring unit 102. Thismakes it possible to sequentially store the retrieved case data in casedatabase 101, so that the number of cases to be retrieved can beautomatically increased.

Also, in the present exemplary embodiment, description has been made bytaking a pair of lungs as an example of a pair of right and left organs.However, the present disclosure is not limited to this example, and isapplicable to other pair of right and left organs such as brains,breasts, and kidneys.

Here, supplemental explanation about the similar case retrieval will bemade. FIG. 12 is a flowchart for showing a position of the similar caseretrieval in accordance with the present disclosure. As shown in FIG.12, images such as CT images of a patient are photographed first (X01),and then an image interpreter interprets an image case from thephotographed images (X02). Then, the image interpreter retrieves similarcases with respect to the interpreted case as necessary (X03), andwrites a report by reference to the retrieved similar cases (X04). Here,the similar case retrieval in accordance with the present disclosurecorresponds to step X03. That is, the similar case retrieval related tothe present disclosure does not fall under the so-called medicalactivity, that is, a process of surgical, curative or diagnostictreatment of human beings, but is equivalent to a kind of informationretrieval technique. Accordingly, the contents of the present disclosurefall under the industrially applicable inventions.

The similar case retrieval apparatus in accordance with the presentdisclosure have been described in the above based on the exemplaryembodiments. However, the present disclosure should not be limited tothe exemplary embodiments. For example, various modifications which anyperson skilled in the art may think of and apply to the presentexemplary embodiments, and other embodiments which may be made bycombining components of different exemplary embodiments should beincluded within a scope of the present disclosure without departing fromthe spirit of the present disclosure.

The above-described similar case retrieval apparatus may specifically beimplemented as a computer system including, for example, amicroprocessor, a read-only memory (ROM), a random access memory (RAM),a hard disk drive, a display unit, a keyboard, and a mouse. A computerprogram is stored in the RAM or the hard disk drive. The microprocessoroperating according to the computer program allows the similar caseretrieval apparatus to achieve their functions. Here, the computerprogram is configured by combining a plurality of instruction codesindicating instructions for allowing the computer to achievepredetermined functions.

Further, a part or all of the components configuring the above-describedsimilar case retrieval apparatus may be implemented as a large scaleintegrated circuit known as a system LSI (Large Scale Integration). Thesystem LSI is an ultra multi-function LSI produced by integrating aplurality of construction parts on a single chip, and specifically acomputer system configured to include components such as amicroprocessor, a ROM, and a RAM. A computer program is stored in theRAM. The microprocessor operating according to the computer programallows the system LSI to achieve its functions.

Furthermore, a part or all of the above-described similar case retrievalapparatus may be implemented as an IC card or monolithic module, whichcan be detachably attached to the similar case retrieval apparatus. TheIC card or the module is a computer system including, for example, amicroprocessor, a ROM, and a RAM. The IC card or the module may includethe ultra multi-function LSI. The microprocessor operating according toa computer program allows the IC card or the module to achieve itsfunctions. The IC card or the module may also be implemented to betamper resistant.

Further, the present disclosure may be regarded as the methods describedabove. Furthermore, the present disclosure may be regarded as a computerprogram for causing a computer to execute the methods, or as a digitalsignal including the computer program.

Further, the present disclosure may be regarded as a form in which thecomputer program or digital signal is recorded in a non-transitory,computer-readable storage medium such as a flexible disk, a hard disk, aCD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray Disc(registered trademark)), and other semiconductor memories. Furthermore,the present disclosure may be regarded as the digital signal recorded inthe non-transitory storage medium.

Further, the present disclosure may be regarded as a form in which thecomputer program or digital signal is transmitted through an electricalcommunications line, a wireless or wired communications line, a networkrepresented by the internet, data broadcasting, or the like.

Further, the present disclosure may be regarded as a computer systemincluding a microprocessor and a memory such that the memory stores thecomputer program and the microprocessor operates according to thecomputer program.

Further, the present disclosure may be implemented in anotherindependent computer system by transferring the program or digitalsignal recorded in the non-transitory recording medium or bytransferring the program or digital signal through the network or thelike.

The present disclosure is applicable to a similar case retrievalapparatus and the like for outputting a similar case with respect to aresult of a diagnosis made by an image interpreter.

What is claimed is:
 1. A similar case retrieval method for a similarcase retrieval apparatus, the similar case retrieval apparatus includinga display and a hardware processor that executes a program and causesthe similar case retrieval apparatus to perform the similar caseretrieval method comprising: receiving, using the hardware processor,image features of partial images including lesion portions; receiving,using the hardware processor, information indicating a distribution ofthe lesion portions is a unilateral distribution in which the lesionportions are distributed unilaterally in either a right organ or a leftorgan; searching, using the hardware processor, similar case data bysearching through image features of images contained in first case datafor the unilateral distribution of lesion portions in the right organand second case data for the unilateral distribution of lesion portionsin the left organ, and determining similar case data based onsimilarities between the image features of images contained in the boththe first case data and the second case data, and the partial imagesincluding the lesion portion, displaying, using the display, an imagefeature of an image contained in each of the first and second similarcase data when the image feature of the image is determined to besimilar to at least one of the image features of the partial imagesincluding the lesion portion, wherein the both the first case data andthe second case data are included in a case database in communicationwith the similar retrieval apparatus, the first case data only includesimages of lesion portions included in the first case data and the secondcase data only includes images of lesion portions included in the secondcase data, and the determining of similar case data includes determininga cosine distance between an image feature vector which is a vectorrepresentation of an image feature of the partial image containing thelesion portion of the interpretation target image and an image featurevector of the partial image containing a lesion portion contained in thecase data, and calculating a sum of the calculated cosine distances as asimilarity.
 2. A similar case retrieval apparatus comprising: a hardwareprocessor that executes a program and causes: a similar case retrieverto (i) receive image features of partial images including lesionportions, (ii) receive information indicating a distribution of thelesion portions is a unilateral distribution in which the lesionportions are distributed unilaterally in either a right organ or a leftorgan, and (iii) search similar case data by searching through imagefeatures of images contained in first case data for the unilateraldistribution of lesion portions in the right organ and second case datafor the unilateral distribution of lesion portions in the left organ,and determining similar case data based on similarities between theimage features of images contained in the both the first case data andthe second case data; and a display for displaying an image feature ofan image contained in each of the first and second similar case datawhen the image feature of the image is determined to be similar to atleast one of the image features of the partial images including thelesion portion, wherein the both the first case data and the second casedata are included in a case database in communication with the similarretrieval apparatus, the first case data only includes images of lesionportions included in the first case data and the second case data onlyincludes images of lesion portions included in the second case data, andthe determining of similar case data by the similar case retrieverincludes determining a cosine distance between an image feature vectorwhich is a vector representation of an image feature of the partialimage containing the lesion portion of the interpretation target imageand an image feature vector of the partial image containing a lesionportion contained in the case data, and calculating a sum of thecalculated cosine distances as a similarity.